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We obtain phase diagrams of regular and irregular finite connectivity spin-glasses. Contact is 
firstly established between properties of the phase diagram and the performances of low density 
parity check codes (LDPC) within the Replica Symmetric (RS) ansatz. We then study the location 
of the dynamical and critical transition of these systems within the one step Replica Symmetry 
Breaking theory (RSB), extending similar calculations that have been performed in the past 
for the Bethe spin-glass problem. We observe that, away from the Nishimori line, in the low 
temperature region, the location of the dynamical transition line does change within the RSB 
theory, in comparison with the (RS) case. For LDPC decoding over the binary erasure channel we 
find, at zero temperature and rate R = 1/4 an RS critical transition point located at p c ~ 0.67 
while the critical RSB transition point is located at p c ~ 0.7450 ± 0.0050, to be compared with 
the corresponding Shannon bound 1 — R. For the binary symmetric channel (BSC) we show that 
the low temperature reentrant behavior of the dynamical transition line, observed within the RS 
ansatz, changes within the RSB theory; the location of the dynamical transition point occurring at 
higher values of the channel noise. Possible practical implications to improve the performances of 
the state-of-the-art error correcting codes are discussed. 

PACS numbers: 89.90+n, 89.70+c,05.50+q 



I. INTRODUCTION 

The survival of ferromagnetic ordering under the dis- 
ruption of frozen random fields Q and the onset of Spin- 
Glass (SG) order in systems characterized by random 
competing interactions Q are two central problems in 
the statistical mechanics of systems with quenched ran- 
domness. When properly rephrased, these two problems 
turn out to relate to core problems in other, quite dif- 
ferent disciplines. Insight into these two closely related 
problems in any given system is obtained from the cor- 
responding phase diagram as a function of the physical 
parameters, such as temperature and disorder strength. 

Finite-connectivity mean- field SG models 0, 0, IE IE 
and their p-spin counterpart Q are important for two 
distinct reasons. Firstly, despite being mean- field in na- 
ture, they are believed to share common properties with 
finite-dimensional spin-glass systems [E El • Secondly, 
the statistical mechanics of SG model systems with fixed 
finite connectivity relates to the study of various hard 
computational problems. In the last two decades the 
analysis of finite-connectivity SG systems has also of- 
fered new tools for understanding the very nature of 
several optimization problems and improve algorithmic 
performances [TTL Ip3 | . In this paper we first obtain the 
phase diagram of finite-connectivity mean-field SG mod- 
els. We then investigate the relation between features 
of the phase diagram and the performance of Low Den- 
sity Parity Check (LDPC) Codes, analyzing the effects 
induced by the Replica Symmetry Breaking (RSB) na- 
ture of solutions obtained for these finite-connectivity SG 
models ^3|. In order to answer these questions, we ex- 
ploit the relation between finite-connectivity SG systems 



and a class of exactly solvable models; we employ an 
efficient computational method that has been shown to 
be quite effective for understanding several features of 
finite-dimensional SG [l4|. 

We will consider a generalized model unifying the 
frameworks for a few distinct problems. 

Particular attention will be given to two special cases: 
a) The Bethe SG which has been extensively studied 
under both Replica Symmetric (RS) and RSB ansatze 
[ESQ an d recently considered in its ferromagnetically 
biased version b) Gallager (LDPC) error-correcting 
codes . While the former is well known within the sta- 
tistical physics community, many in this community are 
less familiar with the latter and its links to the physics 
of disordered systems. 

Reliable transmission of information in noisy condi- 
tions is one of the basic problems of modern communi- 
cation technology. Correspondingly, one of the central 
problems of information theory is channel coding. Meth- 
ods to achieve reliable information transmission rely on 
the introduction of structured redundancy to the origi- 
nal message in order to compensate for corruption due 
to noise. Shannon derived rigorous bounds on the level 
of redundancy (the code rate) required to enable error- 
free communication for a given channel noise |T^ . How- 
ever, Shannon's theorems are non-constructive and do 
not offer a practical coding scheme. Different methods, 
based on various redundancy construction recipes, have 
been proposed over the years [Ig. Despite the fact that 
error-correcting codes are now widely used in a variety 
of applications, current performances of most methods 
are significantly below Shannon's bound. One family of 
error-correcting codes that have been shown to provide 
close-to-optimal performances is that of Gallager's LDPC 
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codes 16]. In these codes the structured redundancy is 
introduced through parity checks of Boolean sums of ran- 
domly sampled message bits. The relation between the 
parity-check error-correcting codes and SG models, and 
consequently between quite different disciplines as statis- 
tical physics and information theory, has been pointed to 
in the seminal work of Sourlas fl9| . 

Decoding in LDPC codes corresponds to a class of SG 
models defined by an underlying lattice geometry that re- 
flects the low density character of the code construction. 
The importance of this very last point, that made the 
above correspondence useful and of practical relevance 
was not understood until recently [20j. Methods devel- 
oped in the statistical mechanics of diluted SG under 
the RS ansatz have been successfully employed to com- 
pute macroscopic properties of such systems for different 
parameter values; reasonable agreement with observed 
decoding bounds have been reported |2lj | . To our knowl- 
edge however, the only phase diagram that has been eval- 
uated for LDPC codes is in the limit of large coordi- 
nation/large multi-spin interaction numbers |22j . which 
also relates to the phase diagram of the Random Energy 
Model (REM) 23]. 

In the following section we introduce the class of Ising 
spin systems to be analyzed and its links to the Bethe SG 
and LDPC decoding problems. This will be followed by 
section lnil where we review the RS equations and the cor- 
responding solutions are obtained using a computational 
method adopted from the study of d-dimcnsional hierar- 
chical models. As part of this section we will present the 
method and briefly discuss the relation between finite- 
connectivity SG and d-dimensional hierarchical models. 
In section llVl we introduce the equations behind the RSB 
theory of finite-connectivity SG studied here and present 
results obtained for the Bethe SG problem and LDPC 
error-correcting codes under two different transmission 
channels: the binary erasure channel (BEC) and the bi- 
nary symmetric channel (BSC). In section we discuss 
the results obtained and point to future research direc- 
tions. 



II. THE MODEL 

The two different models examined in this work stem 
from a unified physical system of X-spin interaction Ising 
model on a lattice of fixed connectivity C. The Hamilto- 
nian describing the system takes the form 

where S £ {±1}^ represents N binary variables, C(p) = 
... ,ik) a set of indices related to interaction p, and 
the corresponding connectivity tensor with the fol- 



lowing properties: 

.fj, f 1 for C(p,) = (ii, ...i K ) 

(*i> •■•>**) | otherwise 

E A (iM,.,i K ) = C Vt . (2) 

(i 2 ,...,ijf ) 

The parameter J is related to the strength of the random 
exchange interaction. 

The various models considered in this work differ in 
the connectivity parameters and type of bias used, which 
corresponds to an additional component of the Hamilto- 
nian JTJ- 

A. The Bethe SG 

The Bethe SG has been extensively studied in the 
past H,Q,|^ and represents a special case of the Hamil- 
tonian Q where one is only considering pairwise inter- 
actions, K = 2. 

Bethe SG with ferromagnetically biased exchange in- 
teraction J, which we will focus on here, has been re- 
cently studied in [l5f. where the distribution of the 
quenched random variable J takes the form 

P(J) = 6(J - 1) + 1^ S(J + 1) , (3) 

and where the parameter p 6 [0, 1] is p = 1 for a ferro- 
magnet and p = for the unbiased Bethe SG. 

B. LDPC error-correcting codes 

We consider a simple communication model whereby 
L bit messages are encoded using LDPC codes and then 
communicated through two channel types: a) The Bi- 
nary Symmetric Channel (BSC) where bits are flipped 
with probability p during transmission and b) The Bi- 
nary Erasure Channel (BEC) where bits are received with 
probability 1 — p and fail to arrive with probability p. 

A Gallager LDPC code is defined by a binary matrix 
H = [A | B], concatenating two very sparse matrices 
known to both sender and receiver, with B (of dimen- 
sionality (N — L) x (N — L)) being invertible; the matrix 
A is of dimensionality (N — L) x L. LDPC codes can be 
employed over various finite fields |24j : we will restrict 
the treatment in this paper to binary symbols. 

Encoding refers to mapping the original L dimensional 
binary message vector £ S {0,1} L to an N dimensional 
binary code word t 6 {0, 1}^ (N > L) by taking the 
product t = G T £, (mod 2), where all operations are per- 
formed in the field {0, 1} and are indicated by (mod 2). 
The generator matrix is G T = [I \ {B~ 1 A) T ] (mod 2), 
where J is the L x L identity matrix, implying that 
HG T (mod 2) = and that the first L bits of t are 
set to the message In regular Gallager codes the num- 
ber of non-zero elements in each row of H is chosen to 
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be exactly K. The number of elements per column is 
then C = (1 — R)K , where the code rate is R = L/N 
(for unbiased messages). We will mostly focus here on 
regular code constructions. 



a,rgma,xgP(S | z). This corresponds to zero temperature 
decoding [28|. 



2. BEC 



1. BSC 

The corruption process that is most intuitively linked 
to physical systems is the BSC, where the encoded vec- 
tor t is corrupted by flip noise represented by the vector 
C € {0, 1} L with components independently drawn from 
P(Q = (1 - p ) 8(Q + p 6(C - 1). The received vector 
takes the form r = G T £ + C (mod 2). 

Decoding is carried out by multiplying the received 
message by the matrix H to produce the syndrome vector 
z = Hr = H£ (mod 2) from which an estimate £ for 
the noise vector can be inferred. An estimate for the 
original message is then obtained as the first L bits of 
r + C (mod 2). 

The Bayes optimal estimator (also known as Marginal 
Posterior Maximiser (MPM)) for noise vector bits is de- 
fined as (j = &rgm&x s . P(Sj \ z). Computing the MPM 
estimate directly is computationally hard, and is typi- 
cally approximated in practice by employing an iterative 
inference algorithm such as belief propagation |2j| . 

The connection to statistical physics becomes clear 
when the field {0, 1} is replaced by Ising spins {±1} and 
mod 2 sums by products |19| . The syndrome vector ac- 
quires the form of a multi-spin coupling = IXje£(^) Ci 
where i = 1, . . . , N and \i = 1, . . . , (N - L). The K in- 
dices of nonzero elements in the row /i of A are given 
by £(/i) = {ii, . . • , ix} and where Au lt ...i K \ is the con- 
nectivity tensor is given in equation J2J). After gauging 
Sj — > SjQ the multi-spin coupling coefficient vanishes 
(JJp = 1) and the parity check component of the Hamil- 
tonian takes the form with J — > oo forcing all can- 
didate vectors to obey the parity checks. This notation 
is used to distinguish the temperature from the parity 
check constraints and to facilitate the finite temperature 
study of error-correcting codes. 

Prior knowledge of the biased noise vector is added 
through a second component to the Hamiltonian 
After gauging, the new term takes the form H\ — 
~FJ2f=i S j(j where F = ^ logt^p) is the Nishimori 
condition [2^, ^ (where (3 = 1), which corresponds to 
the correct prior for the variables S. Decoding at the 
Nishimori condition corresponds to the MPM estima- 
tion |28|. This gives rise to the Hamiltonian 
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(4) 



for the BSC. 

Another important estimator one should mention is the 
Maximum A Posteriori (MAP) estimator defined as £ = 



Codeword bits transmitted over a BEC have a proba- 
bility 1— p of being received (in tact). Encoding is carried 
out similarly to the BSC, but decoding is based on infer- 
ring the codeword itself, rather than the noise vector. 

In the case of the BEC, solution vectors are, 
by definition, codewords and should therefore obey 
HS (mod 2) = 0, where S are the received vectors (with 
a certain number of unknown bits). 

An estimate of the original message is then obtained by 
inferring the missing bits using MPM. Similar practical 
methods to those described in the BSC may be employed 
also in this case. 

The statistical mechanics formulation of the problem 
is similar to that of the BSC, except for the second com- 
ponent added to the Hamiltonian , that takes the form 
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a-p)n^,i 



(5) 



Code performance in the BEC will be considered only at 
the zero temperature limit as in |2jj . 

A review of LDPC codes and their link to statistical 
mechanics has been presented in |30| . 



III. REPLICA SYMMETRIC THEORY 

The analysis carried out here is a natural extension of 
similar calculations done in the past for the Bethe SG [3l| 
model and p-spin Ising models |3j. We therefore briefly 
review the RS theory of the model described by Hamilto- 
nian |0J as both the Bethe SG and LDPC decoding can 
be considered as special cases of the same model. 

To compute the free energy of the problem, consider 
the replica equality, 



Bf= - lim — 
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(6) 



where (2 n )^ t j is the analytical continuation in the in- 
terval n 6 [0, 1] of the replicated partition function [33j. 
Following the standard derivation, e.g. as in 0, Isij . 
and employing the RS ansatz, one obtains the follow- 
ing expression for the replica symmetric free-energy as a 
function of the local field distribution ir(x) (details can 
be found in Appendix A) 



/3/ = A/ 



(i) 



C- 



.K - 1 



K 



-A/ 



(2) 



(7) 
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There are both site and bond contributions, 

C K-l 



set of two coupled equations (cfr. Appendix A), 
, C-i c-i 



J 3=U'=1 i=l <=1 

A/^ = -Ej J Yldasjirfx^J^iix^Jj), (8) = 3/ j JJ day 7r(ay) <5(x - u({a?j}, J)J.(14) 



where Ej£ denotes an average over the two sources of 
randomness in (jJJ, and 



Jf> = log2cosh/3(]>>({x M 4,J J ) + FC 



(i) 



We denote .F^ = ?P 



a({xjj>},Jj), 
a({xj}, Jj) + u({xj}, Jj) 



(9) 



■ wnere 



pu({ Xj },J) = -log[R(+)/R(-)} 
(3a({ Xj },J) = hog[R(+)R(-)} . 



(10) 



The polynomials R(u) are defined via, 

R(a) = 2J exp/3[yicrCTi + y 2 o-iCT2 H yK<rK-i<r], 



(11) 

where {j/i, • ■ • , yx } = {^l, • ■ ■ £k"-i> The above ex- 
pression, in a similar form, has already been obtained 
in pol |. and the equivalence with their expression is 
shown in Appendix A. We prefer the current formula- 
tion because of its manifest simplicity (the two contri- 
butions to the free-energy coming from sites and bonds 
are here in evidence) and because it depends on the func- 
tion a({xj}, Jj), which will play a central role in the RSB 
case, examined in section HVI 

The above writing also introduces the polynomials 
R((j), already well known in the context of hierarchical 
lattices . Evaluating the saddle point equations of the 
above free-energy expression is the basic premise of the 
RS theory, so that the crux of the quen ched randomness 
problem Q lies in the convolution |20| . 

7r(x) = EjX jY[ dXi 3 n ( X ij) -fc{{ x ij}iJ>d)i 

where i = 1, • • ■ , C — 1, j = 1, • ■ ■ , K — 1; and where the 
recursion relations are defined via 

c-i 

n{{ Xij }, j,0 = J2 J i) + F < ■ ( 13 ) 

We can also rewrite for convenience the above convolu- 
tion with the aid of the auxiliary distribution tt(x), as a 
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A. Saddle Point Equations and LDPC Decoding 

To briefly explain the link between LDPC decoding 
and the replica formulation we will restrict this discussion 
to the BSC. The replica analysis as well as decoding in 
other noisy channels follow a similar path [35|. 

Following the derivation of [2l| we write saddle point 
equations for the LDPC decoding problem, considering a 
special case of the Hamiltonian an d the saddle point 
equations (|14|l . where J — > oo is deterministic 



. c-i 



C-l 



tt(x) = Eq J JJ^ dii n(x) S^x — x\ — F(^J 

i—l i—1 
f K-X 

h&) = / n dx 3 6 ( x - u p(i x j})) ■ (1 5 ) 

3 3=1 

Most of the numerical results obtained so far are obtained 
along the Nishimori line (3 = 1. Here we investigate the 
phase diagram at any prior temperature 1//3F, for differ- 
ent choices of the parameters C and K. A typical case we 
will discuss in detail is the one corresponding to C — 3, 
K = 4. 

We will check the consistency between the results ob- 
tained and true decoding experiments, using Belief Prop- 
agation (BP) [3(|, at and away from the Nishimori con- 
dition. We will not review BP in th e paper as it is a well 
known message passing algorithm [25l l37j| with st rong 
links to the Bethe approximation and its variants |38ll39| . 
The iterative algorithm is a microscopic equivalent of 
equations IjlSp. 



B. The Computational Method 

The set of (RS) equations l|14|l have been considered 
and solved in a variety of different ways already. This 
section introduces to a different method. We propose an 
alternative approach to existing methods (e.g. popula- 
tion dynamics or Monte Carlo techniques) which solves 
equations of the form l|12|) efficiently and accurately. The 
main reason why it has not been used to study finite- 
connectivity SG models so far is because of the differ- 
ent context in which it was firstly introduced [40|. The 
method we propose allows to determine the phase dia- 
gram of diluted systems of the type studied here and an- 
alyze the nature and exact location of the phase bound- 
aries. It borrows tools and concepts from a different 
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branch of research, namely the area of hierarchical mod- 
els 0, |4l|, to obtain accurate phase diagrams. The 
iterative method we now describe has a corresponding 
computational effort that scales as N K_1 , N being the 
number of bins introduced and K the multi-spin interac- 
tion parameter. 

Equation i|12|l complicates, after a few iterations, even 
a simple initial local field distribution. The distribution 
-rt{x) and its conjugate distribution n(x) are represented 
via histograms. Each histogram is characterized by two 
quantities, Xij and tt(x), where the latter is the asso- 
ciated probability. Equation Ijl5|l is the convolution of 
(K — 1)C distributions, taking into account the distri- 
bution of the random field P((), contained in the av- 
erage E^(..) = J P(£)d£(..). This is numerically con- 
structed from pairwise convolutions of distributions. For 
a R = 1/4 rate Gallager code , e.g., three pairwise convo- 
lutions are required. A pairwise convolution (or binary 
operation) is achieved as follows: 

(1) The histograms are placed on a grid properly drawn 
at each iteration step of the algorithm. All histograms 
that fall within the same grid cell are combined in such 
a way as to preserve the averages across the domain of 
the local field (and its conjugated) distribution(s). The 
histograms that falls outside the grid, representing a pre- 
determined small fraction of the overall probability, are 
similarly combined into a single histogram. In this pro- 
cedure, a histogram that falls within a narrow band of 
a grid boundary is proportionally shared between neigh- 
boring bins. The gridding is done separately for x > 
and x < 0. 

(2) Two gridded distributions are convoluted according 
to equation (|14|l regenerating the original number of his- 
tograms. 

The convolution of C(K — 1) distributions is achieved, 
from pairwise convolutions, as follows: (1) The pairwise 
convolution contained in the first equation of ifHjl is cy- 
cled once for, e.g., a code rate R = 1/4. (2) The resulting 
distribution is pairwise-convoluted with the noise channel 
distribution P((). (3) The resulting distribution is fur- 
ther self-convoluted according to the second equation in 
(fHj) . This completes the entire process, that is then iter- 
ated until convergence is achieved. Multiple cycling oper- 
ations are also an interesting option being related to, e.g., 
the source compression problem. The flows of the prob- 
ability distribution ir(x) determine the phase diagram of 
the system. Most of our results have been obtained con- 
sidering 125 x 4 independent bins, corresponding to the 
flows of 10 3 quantities (the corresponding number of el- 
ementary operations considered is the number of bins to 
the power of K — 1). Each of the two grids at x > 
and x < have e.g. 125 bins on each side of the average 
value. 

Similar set of equations to ljl2|l have been discussed in 
a variety of different contexts. A very important one is 
that of real-space renormalization-group theory ^(j EH 

E2- 

Eventhough the algebraic form of the convolution cor- 



responding to these two problems is the same, the two 
problems are indeed very different. In the case of hierar- 
chical models the distribution of the exchange interaction 
is studied, while in the problem we are discussing in this 
work one considers the distribution of the local field. This 
reflects the difference in the two underlying geometries. 
The d-dimensional hierarchical lattice include loops that 
are one of the essential characteristics that makes hier- 
archical lattices better candidates, within exactly solv- 
able models, to study the problem of finite-dimensional 
SG 0| ■ The similarity between equations Ijl5|l and those 
related to hierarchical models is indicative of a deeper re- 
lation between the two ( both geometries are hierarchical 
in the sense of ref. an d wm be the subject of future 
work. 



C. The LDPC Phase Diagram - BSC 

The calculated phase diagram for the Gallager LDPC 
code of rate R = 1/4 and a BSC of flip rate p as a func- 
tion of the prior temperature 1/ 'f3F are shown if Fig. ^ 
We report both the location of the dynamical noise tran- 
sition p ( i, defined by the noise level at which suboptimal 
solutions emerge, and critical noise transition p c , where 
the ferromagnetic solution ceases to be dominant, for all 
ranges of temperature 1 / j3F. The phase diagram was cal- 
culated considering the flows of 10 3 quantities dictated 
by the convolution l|12(l . As mentioned before the above 
number should be taken to the power K — 1 to obtain the 
number of elementary operations within, e.g., the second 
equation of l|14fl . 

The location of the dynamical and critical transition 
values can be determined by analyzing the free-energy 
and the complexity as a function of the channel noise level 
for different values of the temperature 1/(3 (Fig. EJ) . We 
will refer noise level that corresponds to the emergence 
of sub-optimal solutions as the dynamical transition. At 
any value p < pd, the computed free-energy JSJ reduces 
to the ferromagnetic free-energy / = — 1 ~ 2p log( : ^ £ ). 
Consider first temperature values in the proximity of the 
Nishimori line. The trajectories exhibit a crossover to 
an unstable distribution first, and then flow to the ferro- 
magnetic sink below ~ 0.1665 (corresponding to a decod- 
ing success) or to a stable non-ferromagnetic distribution 
(corresponding to a decoding failure) above it. A simi- 
lar behavior is observed for any temperature above the 
Nishimori line, the stable distribution being different for 
each values of the temperature f3 < 1. Rather different 
behavior is observed below the Nishimori line, where the 
unstable distribution is a strong coupling distribution, 
meaning that the average local field strength increases 
but the rescaled distribution remains constant. Finally, 
the trajectory flows again, either to the ferromagnetic 
sink or to a fixed local field distribution, which is stable 
under iteration of the recursion (|15fl . 

The computed free-energy, according to the convolu- 
tion l|15fl . is higher than the ferromagnetic free-energy 
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Noise Level, p 

FIG. 1: The phase diagram for the LDPC code of rate R — 1/4 over the BSC channel. The solid and dashed lines represent 
the critical and dynamical transitions, respectively. The inset shows the curvature of the dynamical transition line boundary, 
together with decoding experiments results (square boxes). The dotted curve represents the Nishimori line. The black filled 
dot indicates the location of the dynamical transition at zero temperature assuming the local field distribution to be peaked 
over integer values only. 



for all noise levels pd < p < p c - Above the critical 
noise value p c , the computed free-energy becomes lower 
than the ferromagnetic free-energy. The location of the 
dynamical transition has been determined, at any tem- 
perature, considering the flows, under iteration, of the 
convolution 112(1. while the location of the critical noise 
threshold has been determined from the free-energy (JSJ 
as the point where the computed and the ferromagnetic 
free-energies coincides. In the inset of Fig. ^ we also 
report in square boxes the results of real decoding exper- 
iments obtained as follows. We iterated the probabilistic 
decoding algorithm 25j , at different values of the noise 
channel p, for an ensemble of 10 3 networks of 10 4 and 
2 x 10 4 nodes respectively. The inset shows the calcu- 
lated mean values, the corresponding error being smaller 
than the corresponding symbol size. A precise estimate 
of the dynamical transition point has been obtained con- 
sidering the percentage of decoding success within the 
network ensemble as a function of the noise level, for 
different system sizes, [44|. The decoding experiments 
were performed for different choices of the temperature. 
Along the Nishimori line (see Fig. ^) we find the value 
of pd ~ 0.1665 ± 0.0005 in excellent agreement with the 
reported value for regular R = 1/4 Gallager codes [4^ . 

Notice that the reentrant nature of both the dynamical 
and critical boundaries, as we found solving the convo- 
lution (|14fl . is confirmed by the decoding experiments. 
In the range of noise levels 0.1315 < p < 0.1665, decod- 



ing is possible only in a range of finite temperatures , as 
indicated in the inset of Fig. The boundary between 
decodeable and undecodeable regions reaches its maxi- 
mal value at the Nishimori temperature; zero tempera- 
ture decoding fails at the same noise level. The reentrant 
nature of the critical phase boundary increases below the 
Nishimori line, but eventually weakens as the tempera- 
ture is decreases further, a phenomenon that has already 
been discussed in a variety of different contexts 0, |Z3 • 

The fact that MPM and zero temperature BP/RS de- 
coding do not coincide is not in contradiction with the 
results obtained in ^48'J which indicate that MAP and 
MPM decoding provide the same critical transition point. 
Clearly, zero temperature BP /RS decoding does not pro- 
vide MAP results due to RSB effects. The latter and 
their effect on the phase diagram and on the reentrant 
nature of the phase boundaries will be further discussed 
in section IT*Vl 

An independent calculation has been performed at zero 
temperature. In this case the recursion relations i|13|) 
simplify and one finds limiting expressions for the two 
functions u({xi}, J), a({xi}, J). Implementing numeri- 
cally such asymptotic expressions would require an inde- 
pendent calculation where one explicitly changes the re- 
cursion relations (|13|) to their asymptotic /3 — > oo form. 
The importance of the polynomials R(cr) becomes clear 
at this point. At any temperature considered, we dis- 
tinguish the finite and zero temperature contributions, 
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FIG. 2: The computed free-energy JHJ as a function of the noise level p for different values of /3. The solid line corresponds to 
the free-energy at the Nishimori line (3 = 1, while the dashed lines correspond to the values of f3 = 8/6, 8/4 below the Nishimori 
line and f3 = 8/10, 8/14 above it. For flip rates p larger than the dynamical transition the computed free-energy is higher than 
the ferromagnetic one, given by /p erro = — 1 ~ 2 2p /og(^— E ) (dotted- line). 



corresponding to factorize the largest exponent in the 
sum (llljl , so that the limit of zero temperature is partic- 
ularly simple to handle in our approach and no explicit 
changes are required in the recursion relations i|13|) . We 
observe that at zero temperature the local field distribu- 
tion is not peaked on integers as one might expect. We 
find that the relative weight of the background behind 
the integer-peaked distribution weakens as one consider 
the RSB solution, but restricting the distribution of the 
local fields to integer values, which results in a modifi- 
cation of the recursion relations (|13[1 leads to different 
results. For LDPC (RS) decoding in the BSC this leads 
to an erroneous dynamical transition value of pd — 0.121, 
marked in Fig. ^ by a black filled dot, well below the re- 
ported zero temperature value of pd — 0.1315. Similar 
behavior occurs in the binary erasure channel problem if 
one simply neglects the background distribution, as we 
will discuss in section llVl 

The initial condition ir° (x) = (1 — q)8(x — l)+qS(x+l), 
at q = |, has been used as initial condition for the BSC 
analysis. A dependence of the solution of the convolution 
(|15fl on the initial condition for the local field has been 
observed. We have studied the above convolution H15|) 
for different choices of the initial condition in the form 

n(x) = (1 - q) S(x - 1) + q S(x + 1). (16) 

The choice of q = p, where p is the flip rate of the channel 
results in the phase diagram of Fig. [3J Any choice of 



initial condition with q > p (including the case q = 1/2 
considered above) leads to the dynamical transition line 
(dashed line in Fig.^l, except for the choice q—p which 
leads to a different dynamical transition. The location of 
the critical transition remains unchanged. 

For instance, using the initial condition q = p, the dy- 
namical transition along the Nishimori line is increased 
to pd — 0.1734, well above the known dynamical transi- 
tion point. The physical interpretation of this behavior 
remains to be explained. Since it results from a simple 
change in the initial conditions of i|12|) . one might ar- 
gue that a properly modified decoding algorithm would 
enable us to decode above current pd values; this is a 
possibility that we cannot rule out and further research 
is needed in order to elucidate this point. 

We where able to perform a second set of decoding 
experiments that coincide with the dynamical transition 
line in Fig. [3] but only for noise level values below the 
value pd — 0.1665. The reentrant nature of the dynami- 
cal transition line (the dynamical transition at zero tem- 
perature being pd — 0.158) is reduced in this case. 

D. Irregular Gallager Codes 

For a constant K value one can select non-integer con- 
nectivity values of average C and consider a modified 
form of the convolution (|15f) that involves a linear com- 
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FIG. 3: The phase diagram of LDPC decoding for the BSC with the initial condition 1)16^ . The location of the critical transition 
in unchanged unlike the location of the dynamical transition, that has been observed to increase. We observe, on the Nishimori 
line, a dynamical transition occurring at — 0.1734, well above the ordinary dynamical transition of the regular Gallager code 
of rate R= 1/4. 



bination of terms corresponding to different connectivity 
values. Non-integer values of the multi-spin interaction 
parameter K can also be easily accommodated in 1)1 5[l. 

It has been reported that irregular low-density parity- 
check codes outperform regular constructions |25l |44| . 

We considered a generalized form of the convolution 
(|15fl . (an explicit form is not provided for brevity) where 
both connectivity C and multi-spin interaction K take 
non-integer values such that the rate R = 1 — C /K =1/4 
is preserved. In Fig. 0] we report the dynamical tran- 
sition line for the regular construction C = 3, K = 4 
(R = 1/4) already considered in the previous section 
together with the dynamical transition lines for irreg- 
ular constructions of two values K = 4 and K = 3 and 
C = 2 and C = 3, such that C = 2.9, 2.8, 2.7 (left to 
right) and K ~ 3.87, 3.74, 3.6. The dynamical transi- 
tion value is observed to increase for the irregular con- 
structions. This does not give a direct information on 
the specific pattern of irregularity that one should con- 
sider but provides quantitative information on the aver- 
age connectivity/multi-spin interaction values one should 
investigate further. As for the regular case discussed ear- 
lier, a phase diagrammatic approach to the problem of 
irregular LDPC codes is important to understand the 
reason behind the successful decoding properties of irreg- 
ular LDPC codes. A similar behavior to the one shown 
in Fig. 0] but less pronounced, has also been observed 
for LDPC decoding with the initial conditions l|ltj[) when 
q=p. 



E. Finite connectivity problem with C/K > 1 

We considered the finite-connectivity model Q for the 
values of the connectivity C — 9 and K = 4. Despite the 
fact that it does not correspond to any Gallager LDPC 
code construction it is interesting to consider these con- 
nectivity parameters for two reasons: 1) They can rep- 
resent code constructions in other variants of the LDPC 
family such as Sourlas or MN codes |25j, and as such 
may also be linked to other LDPC-based applications. 
2) They give rise to a SG behavior which is of inter- 
est. In Fig. we report the corresponding phase dia- 
gram obtained considering the convolution (|15fl . An SG 
phase occurs in this case. At low values of the flip rate 
p the convolution (|12|) flows to the ferromagnetic sink. 
In the vicinity of the dynamical transition line (marked 
in Fig as the solid black line) the trajectories under 
iteration of the above convolution cross over to an unsta- 
ble distribution and then either flow to the ferromagnetic 
sink, below the dynamical transition, or to a stable non- 
ferromagnetic distribution. For temperatures below the 
Nishimori line, at high enough values of the flip rate, the 
convolution instead flows to a spin-glass distribution. 



IV. REPLICA SYMMETRY BREAKING 

The role of RSB for the problem (0} is the subject 
of this and the following sections. The one-step RSB 
solution for the Bethe SG, as a special limiting case of 
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FIG. 4: Irregular LDPC phase diagram for the code rate R=l/4 . We show the dynamical transition for different values of 
the connectivity parameters C and K. The solid line corresponds to the dynamical transition line of a regular construction 
with C — 3 and K = 4. The dashed, dotted and thin-dotted lines correspond respectively to C = 2.9, 2.8, 2.7, at constant 
rate R — 1/4. At lower average connectivity values, e.g. along the Nishimori line, the convolution 11511 no longer flows to a 
ferromagnetic sink, but we observe a stable crossover to a distribution with large but finite average, so that successful decoding 
cannot be expected. 




FIG. 5: The dynamical transition line for the finite-connectivity system with C — 9, K — 4. We report the location of 
the dynamical transition, shown now in thick solid lines. At high enough values of the flip rate, the convolution 11511 either 
flows to a stable, asymmetric local field distribution (at high enough temperatures), or to a symmetric SG one (below the 
Nishimori line). Before flowing to the corresponding sink, the flows of the convolution I15H crosses over in the vicinity of either 
an unstable distribution above the Nishimori line, shown as the dotted line, or in the vicinity of a strong coupling unstable 
local field distribution below the Nishimori line. Similar behavior has been observed for the LDPC convolution 115H with the 
ordinary initial condition q = 1/2. In that case the location of the dynamical transition occurs at lower values of the noise 
channel probability. The inset shows the location of the dynamical boundaries together with the Nishimori line. 
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the Hamiltonian 0}, has been studied in detail long ago. 
In this case it is well known that the RS solution is in- 
correct [ijj. Recently, the onset of RSB phenomena in 
some limiting cases of the problem (0} has been consid- 
ered, relying on the variational approximation |9| (for a 
recent formulation see 49] ). Consider the equation for 
the global order parameter <?({c"}), firstly introduced 
for the Bethe SG by Mottishaw, [5| in the general case of 
a multi-spin interaction parameter K, 



s(K» 



Ej X [Tr {a?} g c ~\{an) • • -^(Kr-i}) 



x exp[/3J > a.; it 



1 



'K-l 



(17) 



where a — 1, . . . , n labels different replicas of the sys- 
tem. The above expression holds in general, whether 
replica symmetry is broken or not (see Appendix B for 
the derivation). In the RS approximation one assumes 
that different replicas are equivalent and the functional 
order parameter g({<r a }) depends only on the variable 
a = ^2 a & a - In the limit of n — » 0, can be shown to 
satisfy, 



, K-l 



9(v)=E JX [ ft %c-i( iSj ) d Uj 



K-l 



exp[-cr//3tanh x (tanh(^J) J| tanh(/?Uj - FQ)]. 



(18) 



This equation corresponds to the equation that have been 
written for the Bethe SG, as soon as K = 2 and for van- 
ishing message prior temperature F = Q- A deriva- 
tion of equation (|18f) is found in Appendix B, where we 
show the equivalence of the above expression with the set 
of recursion relations (|12fl . for the local field and conju- 
gate distributions. In the one-step RSB approximation 
one assumes instead that the functional order parameter 
g({(J a }) depends only on the variables om = 
where a — (M, 7) is the replica index, M — 1, . . . , n/m 
and 7 = 1, . . . , m. In this case one finds that the cor- 
responding equation for the functional order parameter 
g({<r a }) becomes 

. K-l n/m 

9 ({a M })=^Ej, c J EE ^T^-HisMj) 



j=l M=l 



Y[ dr M ^™ iaui exp [ - <rMu({r Mj }, J) 

Mj M 

exp [?E a ^ rM ^ ,J ) 

M 



where the normalization is given by 

. K-l n/ 



J j=l M=l 

f dr Mi j™ iSMi cosh m (/5r Mj ). (19) 

Mj j 

The corresponding one-step RSB convolution follows, as- 
suming that the order parameter factorize according to 



9(Wm}) 



n/m 

n f(^) 



(20) 



M=l 



The assumptions behind the factorized ansatz (|20|) . where 
the local field distribution is independent from the intra- 
group index, are discussed in [lOj. In this case the one- 
step RSB convolution (|19|) become 



, C-i 



C-i 



(21) 



tt(x) = E{ I dxiTT(x)s(x — ^ £i — F( 

i=l i=l 
r K-l 

tt(x) = J\f~ 1 Ej / Y\_ dxjir(xj) exp[/j,a({xj}, J)] 
J j=i 

s(x- u({xj}, J)j, 
where the normalization Af is given by 

. K-l 

Af = / ][J dxjTr(xj)exp[fj,a({xj},J)], (22) 
J i=i 

and where fi = m/fl. The above equations are valid, 
under the assumption (|20|l . at finite temperatures and 
arbitrary connectivity values C and K . After its very first 
formulation [lOj the convolution l|21|l has been studied in 
a range of different contexts, from random K-SAT to the 
coloring problem J§jj. Using the identity, 



exp[/ia({xj}, J)] 



2 cosh(w({xj}, J)) 



Ylj cosh(/3xj) 



(23) 



and substituting it in equation (|21|) one recovers the one- 
step RSB equations that have been also derived using 
the variational approach [H I29T |49| . This shows that 
the so called frozen variational ansatz, considered in 
and the factorized ansatz of [Toj are equivalent. Even 
though there exists some degree of arbitrariness [HJ in 
the way one redistributes the re-weighting between the 
two equations Q21|). we prefer the notation that involves 
the exponential of the energy term a({xj}), rather than 
the less obvious (and numerically more difficult to im- 
plement) parallel expression that follows considering the 
right hand side of (|23l) 49, 50] . The simplified expression 
of the free-energy presented in section III was obtained 
in virtue of the same identity l|23|) . and the same degree 
of freedom writin g th e free-energy was already discussed 
for the Bethe SG |Tl| (the right hand side of QEft being, 
at K = 2 the function c(x, J) discussed in ref. (llj). 
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A. The Bethe SG 

We apply the method described in section III to study 
the one-step RSB convolution i|21|) of the Bethe SG, 
which has been extensively studied and recently consid- 
ered also in its ferromagnetically biased version [l^. In 
this case the distribution of the quenched random vari- 
able J is 

P(J) = ^S(J 1) + —^-S(J + 1), (24) 

and where the parameter p G [0, 1] is p = 1 for a ferro- 
magnet and p = for the unbiased Bethe SG. The Bethe 
SG corresponds indeed to the original problem Q at 
K = 2. We will present results obtained at zero temper- 
ature. In the zero temperature limit we find the following 
expression for the free energy 

$(p) = AS (1) - ^A_E (2) , (25) 

where explicit expressions for the site and link contribu- 
tions to the ground state energy are 




cxp ( p max(xicri + x 2 a 2 + J<7\<7 2 ) I , 

V 0"i,02 / 

(26) 

corresponding, at p, = and K = 2, to the RS equa- 
tion © in the zero temperature limit and neglecting 
the prior term in Q). Similar expression have been dis- 
cussed in llj. In the limit of zero temperature one 
finds a(J,x) = \x\ + S Xt o and u{x,J) — JS{x), being 
S(x = 0) — 0, and S(x ^ 0) = sgn(x). In what follows 
we present results obtained under the integer peaked 
ansatz as well as for the full distribution. The relative 
importance of non-integer values in the distribution that 
solves the above convolution 121|) is rather limited in this 
case. The background behind the integer peaked val- 
ues weakens as one consider the one-step RSB case, and 
may disappear in a fully replica symmetry broken sce- 
nario |ll|. In Fig. we show the ground state energy as 
a function of the replica symmetry breaking parameter 
p for different values of the ferromagnetic bias using the 
integer peaked ansatz and in the case of the full local field 
distribution. The ground state energy one extrapolates 
including the background calculation is hi ghe r than the 
one obtained within the three peak ansatz |Toj . In Fig. [7| 
we show the corresponding complexity or configurational 
entropy, that can be obtained as a parametric plot in 



terms of the RSB parameter p, the free-energy f(p) being 
its Legendre transform We observe a positive com- 
plexity for all reported values of p. At higher values of 
the ferromagnetic bias p > p* ~ 0.74 a positive complex- 
ity is not observed anymore. One should also consider 
that the above values should be extrapolated in the limit 
of large number of bins, so that one expects a slightly 
higher value p* ~ 0.75. In Fig. [8] we show the magneti- 
zation for in the replica symmetric and one-step replica 
symmetry broken solution at different values of the ferro- 
magnetic bias. At values p < 0.65 one recovers the results 
known for the unbiased Bethe SG. The solid line indicates 
the magnetization computed in the replica symmetric ap- 
proximation (notice that this is a computed magnetiza- 
tion for a finite number of bins, so that it approaches 
the expected value of pus = 0.75 from below), while the 
square boxes represent the magnetization in the one step 
RSB approximation without considering the background. 
For each value of p, we extrapolated the magnetization 
that corresponds to the value of p where the ground state 
energy has a maximum. Similarly, the dashed line points 
to the results obtained considering the true convolution 
(|12fl . that includes non integer values of the local field. 
The putative size of the so called mixed phase, previ- 
ously discussed 0, if any, is rather small and do not 
agree with previous estimates (we observe the magne- 
tization within the RSB calculation to vanish at values 
of p ~ 0.75, if one considers a large enough number of 
bins within the calculation, implying that the size of the 
mixed phase, if any, is vanishingly small). It should be 
said however that the calculation of [15( considered a 
full one step RSB calculation, in the sense of where 
the factorized approximation is relaxed. To investigate 
this point better, we performed the following: instead of 
implementing the convolution (|12fl within the factorized 
approximation [ljj, we choose a population of 5 x 10 5 dis- 
tributions (including a site dependence). At each step of 
the iteration process we consider pairwise-convolutions of 
local field distributions corresponding to different sites, 
selected at random; after a transient regime, we obtain 
the equilibrium population. Waiting a number of itera- 
tion steps of the order of the population size, according to 
our calculation, the local field distributions become site 
independent, meaning that the factorized approximation 
provides similar results to those of the full RSB ansatz. 
If one compares Fig. [7] with the corresponding plot in 
ref . |15| , where a similar calculation of the complexity is 
presented, we see that the location of the maximum of 
the complexity as a function of p decreases for increas- 
ing p values (see Fig.0, as one might intuitively expect, 
increasing the ferromagnetic bias. 



B. The Binary Erasure Channel 

In the case of the BEC discussed in section II, the form 
of the convolution (|15|) simplifies and the distribution of 
the local field, and its conjugated distribution become bi- 
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FIG. 6: The ground state energy as a function of the replica symmetry breaking parameter for different values (from left 
to right) of the bias p = 0.66,0.68,0.70,0.72,0.74. We also show in thin dotted line the ground state energy including the 
background distribution as in ref.0. 



modal. Despite its simplicity the binary erasure channel 
problem is of practical relevance [521 l5flj . For the BEC 
one writes the convolution IJ15JI as 



ir(x) = p5(x) + (1 - p)S 00 (x) 

%{x) = p8(x) + (1 - p)Soa(x) 



(27) 



where #oo(0 is a delta function located at +oo, p(x) is 
the distribution of the effective local field and p(x) its 
conjugate distribution. The resulting convolution |29| 
involving the two distributions p(x) and p(x), in the one- 
step RSB case, is given by 



. C-i 



C-i 



PO) = / II di 6 { x ~ 

J i=i i=i 
K-l . v 

p{%) = ^ fvN v I JJ dxi p(xi) exp[pa({xi},J)} 



5(x - u({xi}, J)), 



(28) 



where /„ are the binomial coefficients of order K — 1 and 
J\f u the set of normalization coefficients defined by 

M v = J Y]_dx i p(x t )exp[pa({x i },J)}. (29) 

i= 1 

One can readily check that the above convolution reduces 
to the variational approximation 0, |29| using identity 



(J2SJ- We note that the variational ansatz, as already 
discussed for the case of the BSC, coincides with the fac- 
torized ansatz (|2l)|l 0, . We consider an explicit ex- 
pression for the one step RSB free-energy that involves 
the binomial coefficients of order K . Within the factor- 
ized hypothesis [10| , the expression for the free-energy is 
easily computed as a function of the RSB parameter fi. 
We present results for the BEC at zero temperature for 
values of the code rate R=l/4 and R = 1/2 (note that we 
also consider in what follows the integer peak ansatz for 
the local field distribution j2?| in order to compare with 
the general case of non-integer local fields) . In the inset 
of Fig. we show the energy as a function of the erasure 
probability p for the rate value R = 1/2 within the inte- 
ger valued local field approximation. Notice that the inte- 
ger value approximation corresponds to having only three 
bins in our approach of section III. In Fig. llOl we compare 
the results within and without the integer approximation 
for a code rate R — 1/4. The dynamical transition is de- 
fined as the point where sub-optimal solutions emerge 
the corresponding complexity being non zero, as already 
discussed for the BSC channel (see Fig. ITUI). The loca- 
tion of the dynamical transition is also the point where 
the recursion (|21|l flows to a subdominant distribution, 
rather than the ferromagnetic sink. This coincides with 
the point where algorithms devised for real decoding ex- 
periments fail to decode the original message. Within the 
RS framework, we find that the location of the dynam- 
ical transition changes significantly if non-integer values 
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FIG. 7: The complexity E as a function of exp(— ^i) for different values of the ferromagnetic bias p = 0.66, 0.68, 0.70, 0.72, 0.74, 
within the three peak ansatz. 



of the local field are taken into account. Within the three 
peak ansatz the local field distribution is simply defined 
as |2l| 

p(x) = p+S(x - 1) + pS(x + 1) + p S{x) (30) 

The calculation we performed within the integer local 
field approximation (for rates R = 1/2 and R — 1/4 in 
Fig. and Fig. ^| respectively) might suggest to inter- 
pret the point p* ~ 0.4294 for the rate value R = 1/2 as 
the dynamical transition point |29| . If however one com- 
pares (see Fig. ITU)) with the results of the local field dis- 
tribution that now includes non-integer values (the thick 
solid line corresponding to p x >o) we see that the location 
of the dynamical transition is rather different. More- 
over the behavior of the local field probability p± around 
the value p* ~ 0.4294, within the three peak ansatz is 
rather suspicious. Indeed, including non-integer values 
of the local field turns out to be crucial for having a 
quantitative description of the dynamical transition and 
possibly explains the disagreement with numerical ex- 
periments that has been reported lately for the R=l/2 
BEC problem [2jJ. If one avoids the three peak ansatz, 
the nature of the flows dictated by the recursion l|28|l is 
smooth as it always leads to a well-behaved behavior of 
the probability densities p x >o and Px<o- Up to this point 
all considerations are still within the RS approximation. 

Moreover, considering the full convolution (|28|l as we 
do in what follows, the location of the dynamical tran- 
sition in the one-step RSB solution becomes higher than 
that obtained within RS, which arguably implies that one 
may be able to construct the algorithmic analog of the 
RSB recursion (|28|l to improve decoding performances. 



Similarly, within the RSB calculation, the location of the 
critical transition is increased for the rate R = 1/4 con- 
sidered above and found to be very close to the Shannon 
bound value 1 - R = 0.75 (cfr. Fig. lilil) . 
We iterated the convolution (|28|l within the one-step RSB 
approximation described in section VI and computed the 
free-energy as a function of the RSB parameter p for sev- 
eral values of the erasure probability p, above the dynam- 
ical transition point pd — 0.602 as reported in Fig.^JJ In 
this range of p values, the free-energy grows as a function 
of p, dropping to the ferromagnetic free-energy value (be- 
ing /Ferro = f° r the BEC) at some critical p* . At higher 
values of the erasure probability p, the free-energy even- 
tually drops to zero after having reached its maximum 
(see Fig. 1111) . According to the standard recipe of maxi- 
mizing the free-energy with respect to p [3j| , we extrapo- 
late the value of the dynamical transition within the one- 
step RSB calculation, finding the value olp^ SB — 0.621, 
indicated in Fig. EH by the dotted line, well above the 
RS value p^ ~ 0.602. The corresponding complexity is 
shown in Fig. 1121 for different values of the erasure prob- 
ability considered. We observe two branches, a physi- 
cal (right) and an unphysical one (left) as explained in 
ref. |llj| . From the former branch we extrapolated the 
RSB energy value corresponding to the noise channel 
probability. Notice that above the RS dynamical tran- 
sition we do observe, for values of p ~ 0.606, 0.613, 0.62 
that the second branch, that is the relevant one, is not 
present. This can be explained because the correspond- 
ing curves for the free-energy (Fig. Ill|) do not reach a 
maximum and drop instead to the ferromagnetic energy 
value (/ ~ 0) . The dynamical transition within the RSB 
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FIG. 8: The magnetization as a function of the ferromagnetic bias p. The thick solid line correspond to the RS calculation 
obtained with 4096 x 4 bins. Square boxes correspond to the one step RSB calculation if one neglects the background local field 
distribution. Finally the dotted line represents the one step RSB calculation including the background calculation, according 
to equation 12 II and was obtained with 32768 x 4 bins. 
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FIG. 9: For the code rate R=l/2 and a BEC channel we report the local field parameter p± as a function of the erasure 
probability p, at zero temperature, within the integer value ansatz 1301 . The continuous lines represent respectively the 
probability of a negative p_ and positive p+ local field. The inset shows the computed energy as a function of the erasure 
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calculation increases. 

If one considers the free-energy as a function of the 
RSB parameter [i at values of the erasure probability 
slightly higher that the RS critical transition point, one 
finds that the maximum is located at positive energies, 
implying that the critical transition point also changes 
under the one step RSB calculation. In this way fFig. H3|) 
we extrapolated the value of the critical transition, within 
the RSB approximation, to p^ SB ~ 0.7450 ± 0.05. Note 
that Shannon's bound for the specific code rate consid- 
ered is 1 — R = 3/4. In comparison, the critical RS tran- 
sition point p BS — 0.6695. The fact that the dynamical 
and critical transition points increase within the one-step 
RSB calculation might be surprising at first. In the next 
section we will show that a similar situation is observed 
for the BSC at temperatures below the Nishimori con- 
dition [2(|, where it is known that the RS solution is 
dominant |54[ ■ It is also reasonable to believe that the 
above one step RSB calculation is exact for this problem 
[Tl| The above findings concerning the location of 

the critical (RSB) transition seem to support this belief. 

C. The Binary Symmetric Channel 

In this section we return to the BSC I0J and employ 
the same considerations discussed for the BEC in the 
previous section. We consider the convolution l(2*T|l along 
the Nishimori line (3=1. The location of the dynamical 
transition has been discussed in section ITTT1 and we found 
the value of p^ s = 0.1665 within the RS approxima- 
tion, in agreement with probabilistic decoding estimates. 
Not surprisingly |54j , if one looks at the free-energy as a 
function of the RSB parameter \i for values of the BSC 
noise probability p > pd, one finds that the free-energy 
decreases (see the inset in Fig. 114(1 . indicating that the 
RS solution is dominant, so that the dynamical transi- 
tion is the same in the one-step RSB case. p^ s = Pd RSB ■ 
However, at temperatures well below the Nishimori line 
instead, i.e. (3 = 8.0, we observe the following facts: 
above the RS dynamical transition p RS ~ 0.135 the free- 
energy increases as a function of the RSB parameter /z, 
dropping to the corresponding ferromagnetic free-energy 
at some value /i*, before reaching a maximum. This 
means that the second, physical branch of the complex- 
ity is not present, the analysis following the same lines 
as for the BEC calculation. At higher values of the flip 
rate, i.e. p ~ 0.18 the second branch of the complexity 
(see Fig. I15|l appears, indicating a non-zero complexity 
and that we are above the RSB dynamical transition. 
The corresponding complexity for the LDPC decoding 
at p = 0.185 is reported in Fig.^J These findings imply 
that the location of the dynamical transition is improved 
with respect with the RS value. 

The same behavior has been observed, progressively 
reducing in importance at higher values of the temper- 
ature T = 2/8 and T = 4/8, while at T = 1, corre- 



sponding to the Nishimori line, this phenomenon is no 
longer observed. The inset of Fig. ^] shows the com- 
puted free-energy as a function of the RSB parameter /i 
at the Nishimori temperature channel noise p = 0.174. 
We conclude observing that, below the Nishimori line, 
the reentrant nature of the dynamical transition is not 
observed within the RSB calculation. 



V. CONCLUSIONS AND FUTURE 
PERSPECTIVES. 

We introduced a new method to compute the phase 
diagram of finite-connectivity SG model systems and dis- 
cussed the connection with LDPC codes. We extended 
the one-step RSB theory, originally formulated for the 
Bethe SG problem, to the Gallager LDPC decoding for 
two different type of corruption processes, BEC and BSC. 
For the BEC at zero temperature we found that the loca- 
tion of the dynamical transition increases with respect to 
the RS case. Our estimate for the RSB critical transition 
value is a mere one percent away from capacity, differ- 
ently from the RS estimate. The critical and dynamical 
properties of LDPC decoding have also been studied for 
the case of a BSC. The reentrant nature of the LDPC 
phase diagram and the corresponding changes that RSB 
implies in its low temperature properties have also been 
studied. We found that below the Nishimori line the dy- 
namical transition increases within the RSB theory. Pos- 
sible implications to improve the decoding performances 
of the state-of-the-art error correcting codes are one of 
the perspectives we wish to address. We would like to 
construct the algorithmic analog of the convolution H21|) 
in order to improve the decoding performances as pre- 
dicted by the RSB theory. The method we used to deter- 
mine the phase diagrams along this work is very promis- 
ing. It allows to study, without loss of continuity, rather 
different problems, ranging from the d-dimensional hi- 
erarchical SG to finite-connectivity mean-field models. 
In this sense it would be very interesting to apply this 
method to clarify a few open issues. On the one side we 
would like to investigate, within the one-step RSB the- 
ory, the finite temperature phase diagram of the Bethe 
SG in a magnetic field. On the other, it would be inter- 
esting to explore the presence, if any, of RSB phenomena 
in ci-dimensional SG hierarchical model systems. 
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Erasure Probability, p 

FIG. 10: For a code rate R=l/4 and a BEC channel at zero temperature, we report the local field parameters as a function of 
the erasure probability p, for the ansatz 1301 1 shown in thin lines, corresponding to p_, p- + po/2 and po- We then report (thick 
line) the results for the local field probability according to the full convolution 1281 . as it is in the simplest RS approximation 
jj, = 0, corresponding to p x< o, being p x >o = 1 — Px<o- The dashed vertical line corresponds instead to the location of the 
dynamical transition within the one step RSB calculation. The inset shows the computed energy as a function of the erasure 
probability, within the three peak ansatz (thin line) and in the case of the full convolution of the local field being considered 
(thick solid line). 
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FIG. 12: Complexity E(-E) as a function of the Ground State Energy E for different values of the erasure probability p. From 
right to left p = 0.606, 0.613, 0.62, 0.63, 0.64, 0.66, 0.67, 0.68 and p = 0.69 for the code rate problem R = 1/4. 



Appendix A - Equivalence of the RS Equations 



Considering the replicated partition function, 



{2 n ) A ,C,J = E A ,JxTr {s?} x 

■Sf 



(31) 



one can perform the average over the correspond- 
ing underlying geometry in l|31|l introducing an inte- 
gral representation for the finite-connectivity constrain 
d {^2^ — C) in the spirit of 0, defining the set of 
overlaps q ai -- a and conjugated functional order param- 
eter q ai - a m , H3- This leads, under the RS ansatz, 



Qai---otm = j dx n(x)x r ' 

<la 1 ---a m — d,X Tr(x)x T ' 



to the following expression for the free energy, 

A" 



C C 
— In cosh (3 J — —Ej 
K K 

K 



| JJdx/7r(a;;) 



i=i 



1 + tanh(/3J) J| tanh(/3a; i ) 



x In 

+ C J dxdxir(x)n(x) log (l + ta,nh((3x) tanh(/3i)) 

+ C J dyn(x) log cosh(/3x) 

r c c 
- E c / J[ dyiTr(xi) \2 cosh/3(^ xi + F()] . (33) 



1=1 



i=i 



(32) 



The corresponding saddle point equations flU are de- 
rived differentiating the above expression with respect to 
the local field tt(x) and its conjugated distribution 7r(x). 
In order to obtain the free-energy ©, the second and 
third term in l|33|) can be combined via equation (|14fl . to 
obtain 

II + 111= + C^-— -AfW -C^-— - log cosh /3 J 
K K 

- C(K-l) J dx 7r(x)logcosh(/?x). 

(34) 
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FIG. 13: The RS and RSB Energy as a function of the Erasure probability. The Shannon bound corresponding to the rate 
R = 1/4 being 1 — R. The points (bold crosses) are extrapolated from the physical branch of the complexity at E = and 
corresponds to the maximum of the free-energy as a function of the RSB parameter fi. The thin line corresponds to the 
factorized ansatz. 



The fourth term can be written as 



Appendix B -The RSB equations 



Consider the set of coupled equations ( 1141) , 



IV = Af { b 1] + Clogcosh(/3J) 



+ C(K-l) / dx 7r(x)logcosh(/3a;), 



(35) 



. c-i 



C-l 



r(a) = E c J Y[ dx, tt(x) s(x - ^ - Fc) 

z— 1 i— 1 

r(i) = -E/ / d^j ir(xj) s(x — u({xj}, J)V(36) 



where we wrote A/^ 1 ) = A/A + A/ fc (1) and where the 
last term in the free-energy expression above is nothing 

but A/a 1 '' . It is easy to see that several terms cancels out, 
together with the first term in equation l|33l) . to obtain 
equation ijHJ. Similar simplifications and their interpre- 
tation were discussed already in the case of the Bethe SG 

El 



For the case of K = 2, with no random bias, correspond- 
ing to the Bethe SG problem, let us define the Laplace 
transform of the conjugated local field distribution as 



(37) 



g(a) = / dx e~ ax %{x). 
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FIG. 14: The Gallager free-energy as a function of the Replica Symmetry Breaking parameter /j, for different values of the noise 
channel p and temperature (the inset showing the free-energy at the Nishimori temperature for a flip rate p = 0.174, above the 
dynamical transition point.) 



Substituting the first equation of l|36[) into the second 
and then into ( 137ft , one finds 

g(a) — J ds S(s + ia) Ej ^ J Y[ d&i ™(xj) x 



exp 



c-i 



tanh _1 (tanh(/3J) tanh(/3 ^ % + @ F )) 



(38) 



which is the expression for the global order parameter 
found in and corresponds to expression ijTHIl for K = 2. 
The expression for arbitrary K values of the multi-spin 
interaction simply involves multiple convolutions of the 
Mottishaw global order parameter. The meaning of the 
global order parameter (|37|l can be understood observing 
that, e.g. in the case of the Bethe SG considered here (in 
a uniform magnetic field) , the partition function is given 
by 



(Z n ) =Tr {CT?} exp 



(39) 



where 



9({<}) = Ej[Tr {a?} (g({an)) 
exp +/3F]>>? 



c-i 



(40) 



The subscript m indicates the dependence of the global 
order parameter on the sub-shell tree structure. If one 
is interested in the thermodynamical limit, it is expected 
that the fixed point of the above recursion determines the 
equilibrium properties of the system. A delicate point 
is the choice of the boundary conditions which are well 
known |ll| to play an important role in this problem. 
The corresponding extension of equation (|40|l to the case 
of arbitrary K values has been reported in equation l|17[l . 
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FIG. 15: The Gallager complexity for a BSC noise level of p = 0.185, at temperature (3 = 8.0, below the Nishimori line. The 
inset shows the corresponding free-energy as a function of the RSB parameter p. 



